Image-based View Synthesis

We present a new method for rendering novel images of exible 3D objects from a small number of example images in correspondence. The strength of the method is the ability to synthesize images whose viewing position is signi cantly far away from the viewing cone of the example images (\view extrapolation"), yet without ever modeling the 3D structure of the scene. The method relies on synthesizing a chain of \trilinear tensors" that governs the warping function from the example images to the novel image, together with a multi-dimensional interpolation function that synthesizes the non-rigid motions of the viewed object from the virtual camera position. We show that two closely spaced example images alone are su cient in practice to synthesize a signi cant viewing cone, thus demonstrating the ability of representing an object by a relatively small number of model images | for the purpose of cheap and fast viewers that can run on standard hardware. Copyright c Massachusetts Institute of Technology, 1997 This report describes research done within the Arti cial Intelligence Laboratory and the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. This research is sponsored by grants from ONR under contracts N00014-96-1-0342 and N00014-93-1-0385; by a grant from ONR/ARPA under contract N00014-92-J-1879 by the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program) and by grants ACTS AC 074 and VANGUARDUS-IS94-00120.